• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    VIDEO DEVICE FOR PROCESSING A VIDEO SIGNAL, VIDEO SIGNAL PROCESSING SYSTEM, METHOD OF PROCESSING A VIDEO SIGNAL AND COMPUTER PROGRAM PRODUCT TO PROCESS A VIDEO SIGNALA video signal is processed on a video device (50). The video signal transfers video data representing either the three-dimensional [3D] video content formatted according to a 3D format or 2D content formatted according to a 2D format. The video signal has a 2D frame and a 2D format control structure to be compatible with existing distribution systems. The device has a processor (52) to provide a 3D status signal indicative of the video signal format. The processor determines a respective format score for at least one of possible 3D formats by processing video data according to the respective predetermined format properties to derive and compare the respective 3D subframes, and establish the 3D status signal based on an evaluation of the format score to indicate the format of the video signal. Advantageously, the 3D format is automatically detected and a 3D display can be controlled in this way.
    公开号:BR112012019612A2
    申请号:R112012019612-0
    申请日:2011-02-02
    公开日:2020-07-14
    发明作者:Wilhelmus Hendrikus Alfonsus Bruls
    申请人:Koninklijke Philips Electronics N.V.；
    IPC主号:
    专利说明:

    VIDEO DEVICE FOR PROCESSING A VIDEO SIGNAL, VIDEO SIGNAL PROCESSING SYSTEM, METHOD OF PROCESS A VIDEO SIGNAL AND PROGRAM PRODUCT FROM COMPUTER TO PROCESS A VIDEO SIGNAL
    FIELD OF THE INVENTION The invention relates to a video device for processing a video signal, the device comprising receiving means for receiving the video signal comprising video data representing both three-dimensional [3D] video content formatted according to a format 3D or two-dimensional [2D] video content formatted according to a 2D format, the 3D format having at least two 3D subframes to constitute a single 3D frame and one of a set of possible 3D formats.
    The invention further relates to a method of processing a video signal comprising receiving the video signal comprising video data representing the three-dimensional [3D] video content formatted according to a 3D format or a two-dimensional [2D] video content formatted according to a 2D format, the 3D format having at least two 3D subframes to constitute a single 3D frame and one of the set of possible 3D formats.
    The invention further relates to a video signal and a computer program product.
    The invention relates to the field of transferring 3D video data through a 2D video data signal format.
    BACKGROUND OF THE INVENTION Devices for generating two-dimensional (2D) video data are known, for example, video servers, transmitters, or creative devices. Currently, 3D augmented devices to provide three-dimensional (3D) image data are being proposed
    Similarly, video devices for processing 3D display video data are being proposed, such as, for example, optical disc players (eg Blu-ray Disc; BD) or set-top boxes that produce received digital video signals.
    The video device must be coupled to a 3D display device such as a TV set or monitor.
    Video data can be transferred from the device through a suitable interface, preferably a high-speed digital interface such as HDMI.
    The 3D display can also be integrated with the video device, for example, a television (TV) having a receiving section and a 3D display.
    WO2009 / 077929 describes approaches that could be taken for the transition between 2D and 3D.
    A 3D video signal has associated video information and playback information, the associated video information and playback information being organized according to a playback format.
    The video information can comprise a primary video stream for 2D display, and an additional information stream to allow 3D display.
    The associated playback information comprises display information indicating the possible display types.
    The display information is processed at a receiver to determine that both 2D display and 3D display are possible.
    A playback mode is established to determine whether video information should be displayed in 2D or 3D mode.
    WO2006 / 018773 describes a system for detecting a vision mode based on an input video signal.
    The video signal can be a 3D video signal containing multiple views.
    Views are arranged in an array of pixel values, whose pixel values must be mapped to a respective structure of data elements corresponding to the pixel positions in a multi-view display. A dedicated 3D video signal is used to transfer the pixel values of the respective views, and the number of views is detected by the receiver.
    SUMMARY OF THE INVENTION A problem with MWO2009 / 077929 is that the transitions between 3D and 2D playback are based on the availability of 3D signaling in the video input signal. However, 3D formats can be mapped to 2D format video signals to be compatible with existing distribution systems for video signals and / or storage media. Due to the lack of signaling in the existing 2D signal format, the user has to manually select the appropriate mode to transform the video signal into 3D.
    It is an objective of the invention to provide a system to transition between 3D and 2D in a more convenient way.
    For this purpose, according to a first aspect of the invention, the device as described in the opening paragraph comprises a processor to provide a 3D status signal indicative of the video signal format, the processor being arranged to determine the respective format scores for a number of possible 3D formats by processing the video data according to the respective predetermined format properties to derive and compare the respective 3D subframes, which determine said respective format scores for said number of possible 3D formats to be arranged in a predetermined order, and establish the 3D status signal based on an assessment of the respective format scores to indicate the format of the video signal, when the assessment of the respective format scores provide a predetermined level of confidence, at which the signal video has a 2D frame and a 2D format control structure, the 3D subframes being generated through a spatial subsampling format and photography elements subsampled from the 3D subframes being arranged in the 2D frame of the video signal.
    For this purpose, according to another aspect of the invention, the method of processing a video signal comprises providing a 3D status indicative of the format of the video signal based on the determination of the respective format scores for a number of possible 3D formats processing the video data according to the respective predetermined format properties to derive and compare the respective 3D subframes, which determine said respective format scores for said number of possible 3D formats to be arranged in a predetermined order, and establish the status 3D based on an evaluation of the respective format scores to indicate the format of the video signal, when the evaluation of the respective format scores provides a predetermined level of confidence, in which the video signal has a 2D frame and a control structure 2D format, 3D subframes being generated through a spatial subsampling format and elements of photo sub-samples of the 3D sub-frames being arranged in the 2D frame of the video signal.
    The measures have the following effect. The video signal arriving at the input is analyzed by the video device to determine a 3D status signal, the 3D status being either a 2D status or a 3D status indicating a format from a set of possible 3D video formats. The video device provides the 3D status signal to control a 3D video display, that is, to establish the operational mode to correctly transform the video signal. The analysis is based on the determination of a format score for the respective 3D formats, that is, assuming that the signal contains video data according to the respective 3D video format, the corresponding 3D subframes are derived from the signal. For example, both 3D subframes are supposed to be placed side by side in a 2D frame. Subsequently, the 3D subframes, for example, a left frame and a right frame, are derived from the signal and compared, that is, analyzed to verify that both supposed 3D subframes in fact have the shape properties corresponding to the 3D subframes. For example, for a table E and D a correlation is calculated, which should be relatively high because the same content is present in both 3D subframes although seen from a slightly different angle of view. Subsequently, the format scores are evaluated, for example, to a predetermined limit. Based on the evaluation of the 3D formats they can score reliably high, and then the 3D status signal is correspondingly established to indicate the format of the video signal. If none of the 3D formats has a sufficiently high score, a 2D video signal is assumed and the status is correspondingly established. Advantageously, the actual mode of a 3D display, for example, a 3D television set, can be automatically controlled based on the 3D status signal.
    The invention is also based on the following recognition. As consumers get used to viewing in 3D, there will be a need to transfer video signals through existing distribution channels, for example, transmission networks or video storage media. In practice, less degradation in resolution appears to be acceptable and content providers can package their 3D content in existing 2D video signal formats by configuring the 3D subframes in the 2D frame. The inventors saw that it is convenient to automatically detect such a specially formatted 3D signal, which cannot load the control data by signaling the 3D format, because inherently the video signal format must remain in the existing 2D format. Although several provisions of the 3D subframes can be used, still a detection of the 3D format seems to be possible based on the first assumption that a respective 3D video format was used and subsequently analyze the supposed 3D subframes for that format.
    Advantageously, based on the current low relative cost of video processing power, analyzes are possible in real time within a time sufficiently short for the user to barely notice the delay in switching from 2D or 3D video mode respectively.
    In one embodiment, the set of possible 3D formats comprises at least one spatial subsampling format to generate the 3D subframes and predetermined format properties - they comprise configuring the subsampled photo elements of the 3D subframes in the 2D frame of the video signal. Spatial subsampling reduces the number of pixels, that is, the resolution, in one or more spatial directions. Advantageously, 3D subframes require a lower number of pixels and can be embedded in a 2D frame (full resolution). The dispositions of 3D subframes spatially sampled in various 3D formats (for example, side by side or top to bottom) are assumed and a respective format score is calculated.
    In an embodiment to determine the respective format scores for a number of 3D formats from the set of possible 3D formats are arranged in a predetermined order, and the 3D status signal is established when the evaluation of the format scores provides a predetermined level of confidence . Advantageously, a high score from a 3D format that is expected is found more quickly.
    In an embodiment to determine the respective format score it comprises calculating a correspondence between the 3D subframes through at least one correlation calculation between the 3D subframes; calculating an average of differences between the 3D subframes; calculation of color properties of the respective 3D subframes to detect a depth data subframe.
    A correlation or having low average differences between both 3D subframes is expected to match the left and right 3D subframes, where the color properties for a depth map as a 3D subframe are significantly different (usually the depth data does not contain color ). In one embodiment at least one of the possible 3D formats comprises left and right 3D subframes [E and D] arranged in the 2D frame according to a left / right polarity, and the processor is willing to, when determining the format score, determine a polarity score based on a predetermined depth distribution occurring in the 3D frame, and establishing 3D status includes establishing a left / right polarity status signal based on an evaluation of the polarity score.
    The depth in the 3D frame can be derived from the disparity values, the actual depth values on a depth map, or an appropriate estimate based on the 3D subframes.
    Detecting the presence of 3D subframes may also require detection whose subframe is left and whose subframe is right.
    If the subframes are interchanged, there is a strong deformation of the depth information in the 3D image.
    Assuming a predetermined distribution of the corresponding depth values or disparity values, a polarity score is determined. Advantageously, the 3D display will be provided with the correct left and right polarity status. In one embodiment, the processor has detector means for comparing the respective 3D subframes by at least detecting a vertical black mask at the vertical edges of the 3D subframes; detect a horizontal black mask at the horizontal edges of the 3D subframes. Based on the presence of a black mask, the respective 3D subframes can be reliably detected.
    According to another aspect of the invention, the video signal comprises video data representing either the three-dimensional [3D] video content formatted according to a 3D format or the two-dimensional [2D] video content formatted according to a 2D format , the video signal having a 2D frame and a control structure of a 2D format, the 3D format having at least two 3D subframes to constitute a single 3D frame and being one of the set of possible 3D formats, the video data having at least at least one vertical black mask at the vertical edges of the 3D subframes, while the aspect ratio of the video content does not require vertical black bars; a horizontal black mask at the horizontal edges of the 3D subframes, while the aspect ratio of the video content does not require horizontal black bars; to allow detection of the black mask to determine the 3D format. Advantageously, based on the presence of the black mask, the respective 3D subframes can be reasonably detected.
    Other preferred embodiments of the method, video and signal devices according to the invention are given in the pending claims, disclosure of which is incorporated herein by reference.
    BRIEF DESCRIPTION OF THE DRAWINGS These and other aspects of the invention will be apparent and further elucidated with reference to the achievements described by way of example in the following description and with reference to the accompanying drawings, in which Figure 1 shows a system for displaying 3D image data, Figure 2 shows a 2D video frame, Figure 3 shows a 3D format side by side, Figure 4 shows a 3D format from top to bottom, Figure 5 shows a 3D format of image and depth, Figure 6 shows a processor for automatic detection of a 3D format, Figure 7a shows a depth map based on a correct left / right polarity, Figure 7b shows a depth map in a wrong left / right polarity, Figure 8 shows a depth analysis in horizontal fields, and Figure 9 shows a depth analysis in vertical fields. In the Figures, the elements that correspond to the elements already described have the same numerical references.
    DETAILED DESCRIPTION OF THE ACHIEVEMENTS It is realized that the current invention can be used by any type of 3D display that has a depth range. Video data for 3D displays is assumed to be available as electronic video data, usually digital. The current invention refers to such image data and manipulates the image data in the digital domain. There are many different ways in which 3D images can be formatted and transferred, called 3D video format. Some 3D formats are based on using a 2D channel to load stereo information as well. This document focuses on 3D formats using a 2D format signal to be compatible with the existing 2D distribution.
    Figure 1 shows a system for displaying three-dimensional (3D) image data, such as video, graphics or other visual information. A source device 40 transfers a video signal 41 to a video device
    50. The source device provides the video signal based on a video data input 43 available from a storage system, 3D cameras, etc. The video signal 41 can be a 2D video signal or a 3D video signal. This document focuses on transferring 3D video content through a signal that is formatted according to a pre-existing 2D signal format, for example, to be compatible with existing distribution channels. In such a 2D formatted video signal, the 3D format defines the location and structure of the 3D video data. Consequently, the video data represents either the three-dimensional [3D] video content formatted according to a 3D format or two-dimensional [2D] video content formatted according to a 2D format. In particular, the video signal has a 2D frame and a structure control of a 2D format, where in the event the 3D format is used, the video data has at least two 3D subframes to constitute a single 3D frame. In fact, several different 3D formats are used, and the video signal contains the structure of one of a set of possible 3D formats. Several examples of 3D formats are discussed below with reference to Figures 2-5.
    The source device can be a server, a broadcaster, a recording device, or a production and / or creation system to manufacture recording carriers like the Blu-ray Disc. Blu-ray Disc supports an interactive platform for content creators. For stereoscopic 3D video there are many formats. The main formats are stereo and image-plus-depth formats. Of these again there are many possible ways in which the content can be formatted to be suitable for use with new and existing 3D displays and distribution formats. More information about the Blu-ray Disc format is available on the Blu-ray Disc association's website on a paper in the audiovisual application format. http: //www.blu- raydisc.com/Assets/Downloadablefile/2b bdrom audiovisualappli cation 0305-12955-15269.pdf. the production process further comprises the steps of deriving the physical pattern of the marks on the tracks incorporating the 3D video signal including the depth metadata, and subsequently molding the recording carrier material to provide the tracks of the marks on at least one layer of storage.
    In one embodiment the source device has a processing unit 42 for modifying the video data from the 3D video input to increase the detection of the 3D video data that is transmitted via the 2D formatted video signal, as explained below.
    The video device 50 is coupled with a 3D display device 60 to transfer a 3D display signal 56. The 3D video device has an input unit 51 for receiving the video signal. For example, the device may include an optical disc drive 58 coupled to the input unit to recover the video signal from an optical recording carrier 54 such as a Blu-ray or DVD disc. Alternatively, the device may include a network interface unit 59 for attaching to a network 45, for example, the internet or a transmission network, such a video device being commonly called a set-top box. The video device can also be a satellite receiver, media player, personal computer, mobile device, etc. The video device has a processor 52 coupled to input unit 51 for processing the video signal. The processor provides a 3D status signal 57 indicative of the video signal format. The 3D status is either a 2D status or a 3D status indicating a 3D format from a set of possible 3D video formats. The processor is arranged to determine a respective format score for at least one of the possible 3D formats by processing the video data according to the respective predetermined format properties. The video signal is analyzed to calculate the format scores for the respective 3D formats, that is, assuming that the signal contains the video data for the respective 3D video format, the corresponding 3D subframes are derived from the signal. Thus the processor derives the respective 3D subframes, and establishes the 3D status signal based on an evaluation of the format score to indicate the format of the video signal. The video device provides the 3D status signal to control a 3D video display, that is, to establish the operational mode to correctly transform the video signal. An exemplary embodiment of processor 52 is described with reference to Figure 6.
    In one embodiment the video device has a detector 53 for detecting properties of the 3D format signal at the edges of the 3D subframes. For example, the detector can detect a vertical black mask on a vertical edge of the 3D subframes, or a horizontal black mask on a horizontal edge of the 3D subframes. Relatively wide black bars may be present in the video data due to a mismatch of the 2D frame aspect ratio and the active video area, for example, a film having a 2.35: 1 aspect ratio in a video frame of 16: 9.
    Such wide black bars can be easily detected, for example, in a 3D format from top to bottom as explained below. In this way, the detection of wide black bars to detect an aspect ratio is known, for example, from the US patent document 5,686,970.
    The detector is coupled to processor 52 to generate the 3D status signal, and can be physically integrated with processor 52.
    In one embodiment the video signal comprises video data representing either 3D content formatted according to a 3D format or 2D content formatted according to a 2D format, the video signal having a 2D frame and a control structure of a format 2D, the 3D format having at least two 3D subframes to constitute a single 3D frame and being one of a set of possible 3D formats, the video data having at least one of a vertical black mask at the vertical edges of the 3D subframes, while the aspect ratio of video content does not require vertical black bars; a horizontal black mask on the horizontal edges of the 3D subframes, while the aspect ratio of the video content does not require horizontal black bars. It is noticed that the black mask is added to the video data not to correct any aspect ratio mismatch, but to allow detection of the black mask to determine the 3D format. The black mask is now a small black bar of one or more pixels in the video area. It is realized that the matte may be small enough to fall within a boundary area of the video frame that is not normally displayed, and is called an overscan area. The black mask can be applied to the top and bottom edge or the left and right edge of the video area. Alternatively, a black mask can be applied to only one side border, for example, the border on which both the 3D subframe will be adjacent when located in the 2D frame of the 2D formatted video signal.
    In one embodiment, detector 53 is arranged to detect the vertical black mask on a vertical edge of the 3D subframes, or a horizontal black mask on a horizontal edge of the 3D subframes, as intentionally added to the video signal defined above.
    Relatively small bars have been included in the video data of the 3D subframes to increase the automatic detection of the 3D video data that is transferred in the 2D formatted video signal.
    The detector derives the specific edge area of the 3D subframes that is assumed to contain the black mask according to the respective 3D video format from the video data while taking into account any processing, such as subsampling, prescribed by the respective 3D format on the coding side.
    In one embodiment, the black levels of the black mask could have different values (for example, 0 and 4) for the left and right 3D subframes.
    Both values will be substantially black when viewed on a display.
    This property can be used to further assist in polarity detection.
    In one embodiment, the processor is arranged to generate a display signal 56 to be transferred via an output interface unit 55 to the display device, for example, a display signal according to the HDMI standard, see “High Definition Multimedia Interface; Specification Version 1.3a of Nov 10 2006 ”available at http://hdmi.org/manufacturer/specification.aspx. processor 52 is arranged to generate the image data included in display signal 56 for display on display device 60. The display signal can be formatted according to the existing 2D signal format, and the status of the 3D signal can be supplied separately, for example, through a separate interface of the 3D display device to control the 3D video display, that is, to establish the operational mode to correctly transform the video signal.
    In one embodiment, the status of the 3D signal can be incorporated into the 3D display signal 56, for example, in a control signal or control data frame. The output interface unit (55) constitutes the transmission means for transmitting a 3D digital display signal, the 3D display signal comprising the 3D video content and the control data indicative of the 3D status signal. In a practical embodiment, the display signal is provided with 3D signaling according to the HDMI 1.4 standard. The 3D display device 60 is for displaying 3D image data. The device has an input interface unit 61 for receiving the display signal 56 which can include the 3D video data transferred from the video device 50. The transferred video data is processed in the processing unit 62 for display in a 3D 63 display, for example, a lenticular or double LCD. The display device 60 can be any type of stereoscopic display, also called a 3D display, and has a range of display depth indicated by the arrow 64.
    In an embodiment of the 3D display device 60 the processing of the video signal and the detection of the 3D and 2D formats is performed on processor 62 in the display device. Video data is transferred via display signal 56. Format detection is performed locally on the display device. The processing unit 62 now performs the function of providing the 3D status signal to generate the display signals either in 2D or 3D mode that are directly coupled to the 3D display. Processing means 62 can be arranged for corresponding functions as described for processor 52 and / or detector 53 in the video device.
    In one embodiment, the video device 50 and the display device 60 are integrated into a single device, in which a single set of processing means performs said 2D / 3D format detection function. The 3D status signal 57 is provided internally to directly control the built-in 3D video display.
    Figure 1 further shows the recording carrier 54 as a carrier of the video signal carrying a 3D format. The recording carrier is in disk format and has a strip and a central hole. The strip, consisting of a series of physically detectable marks, is arranged according to a spiral or concentric pattern of times substantially constituting parallel bands in an information layer. The recording carrier can be optically readable, called an optical disc, for example, a CD, DVD, or BD (Blue-ray disc). Information is represented in the information layer by optical detectable marks along the strip, for example, reliefs and gaps. The strip structure also includes position information, for example, headers and addresses, to indicate the location of the information units, usually called information blocks. Recording carrier 54 carries the information representing the digitally encoded image data, for example, encoded according to the MPEG2 or MPEG4 encoding system, in a predefined recording format such as the DVD or BD format.
    In various embodiments, processor 52 and detector 53 in the video device are arranged to perform the following functions as described in detail below.
    In one embodiment, a method provides a video signal, which comprises video data representing either the three-dimensional [3D] video content formatted according to a 3D format or two-dimensional [2D] video content formatted according to a 2D format, the video signal having a 2D frame and a control structure of a 2D format, the 3D format having at least 3D subframes to constitute a single 3D frame and being one of a set of possible 3D formats, the video data having at least one de - a possible vertical black mask on a vertical border of the 3D subframes, while the aspect ratio of video content does not require vertical black bars; - a horizontal black mask on a horizontal border of the 3D subframes, while the aspect ratio of the video content does not require horizontal black bars; to allow detection, the black mask to determine the 3D format.
    In another embodiment, the method comprises the step of making a recording carrier, the recording carrier being provided with a banner or marks representing the video signal.
    As a product, a recording carrier 54 is provided with a range of marks comprising the above video signal via the above manufacturing method.
    Figure 2 shows a 2D video frame. The Figure shows an example of 2D video content in a 2D video frame indicated by a dotted line 21. The same video content, but in a 3D form, is also used as an example of 3D formats in Figures 3-5. It is noticed that the 2D frame is encoded in a 2D video signal according to one of the several known 2D formats. Encoding can include compression according to MPEG2 or MPEG4 as is well known in the art.
    Figure 3 shows a 3D format side by side, more indicated as SBS.
    The figure shows an example of 3D video content consisting of a left frame E 31 and a right frame D 32 arranged side by side in the 2D video frame 21. Figure 4 shows a 3D format from top to bottom, additionally indicated as TB .
    The Figure shows an example of 3D video content consisting of a left frame E 33 located in the top half of the 2D frame 21 and a right frame D 34 arranged located in the bottom half of the frame 2D 21. A different 3D format based on two views using a 2D image and an additional depth image D, a so-called depth map, which conveys information about the depth of objects in the 2D image.
    The format is called image + depth (2D + D) is different in that it is a combination of a 2D image with a so-called “depth”, or disparity map.
    This is a gray scale image, where the gray scale value of a pixel indicates the amount of the disparity (or depth in the case of a depth map) for the corresponding pixel in the associated 2D image.
    The display device uses the disparity, depth or parallax map to calculate additional views taking the 2D image as an input.
    This can be done in a variety of ways, in the simplest form it is a matter of transporting the pixels from the left or right depending on the disparity value associated with those pixels.
    It is noticed that in 2D + D format, other depth information can be included as occlusion and / or transparency.
    The document entitled "Depth image based rendering, compression and transmission for a new approach on 3D TV" by Christoph Fehn presents an excellent overview of the technology (see http://iphome.hhi.de/fenn/Publications/fehn EI2004.pdf Figure 5 shows a 3D image and depth format, additionally indicated by 2D + D. The Figure shows an example of 3D video content consisting of a 2D 35 frame and a D 36 depth frame placed side by side in the frame. 2D video frame 21. The depth and 2D frame can also be arranged in a top-down configuration similar to Figure 4. Other 3D formats arranged in a 2D formatted video signal will be discussed later.
    The following section deals with the 2D formatted video signal that contains 3D video content according to a 3D format, such as SBS, TB or 2D + D, will be discussed. The list below shows some other subsampling methods and 3D formats for stereoscopic video.
    - Interleaved line (LI) —- Interleaved column (CI) - Checkerboard (CB), also called quincunx * Checkerboard Side by Side (CBS), like the checkerboard, but storing E & D samples as in SBS for better compression.
    An example of CB is described in patent document US2005 / 0117637.
    Assuming that a particular 3D format has been used, and comparing the possibilities using techniques such as motion / disparity compensation, correlation, calculation of absolute mean differences (MAD) etc., the real 3D mode is automatically detected. For the different sampling methods, the system below uses a corresponding method to detect the format. The first step in the method is deriving 3D subframes from the 2D format signal according to the arrangement and / or interleaving of the respective 3D format. For example, for the LI arrangement, the method reconstructs E and D based on the respective lines. Subsequently the alleged 3D subframes are analyzed to determine whether the properties are as expected. If so, a 3D status signal is established to indicate the respective 3D format.
    Figure 6 shows a processor for automatic detection of a 3D format. The Figure shows an exemplary embodiment of processor 52 having a parallel arrangement for analyzing 2 possible 3D formats. In the upper section of the Figure a video input signal 600 is coupled to a division unit of the 3D subframe SPH 601 to divide the 2D frame in the input signal into two (or more) 3D subframes according to a first 3D format. In the example, the SPH unit has the function of dividing the frame horizontally according to a side-by-side arrangement (SBS) of 3D subframes. An E 602 memory unit stores the pixels in the left 3D subframe, and a D 603 memory unit stores the pixels in the right 3D subframe. Units E and D 602, 603 can only store incoming pixels or can increase the sampling of video data to full resolution according to the respective 3D format that is assumed to be used. Better results should be expected for full resolution based on increased sampling, because the video data on one encoder side has been reduced to sampling according to the respective 3D format, for example, in the horizontal, vertical, or quinconium direction, which is now compensated.
    Subsequently the 3D subframes are coupled to a comparison section to calculate a correspondence between the 3D subframes. In carrying out a correspondence calculation unit MAD 605 is provided, which performs the calculation of average absolute differences between the 3D subframes. The average absolute difference of each (or a subset of)
    corresponding pixels in the 2 parts is calculated. Alternatively or additionally, other correspondences between the 3D subframes can be estimated, for example, calculating the mean square differences (MSD), calculating a correlation between the 3D subframes, or calculating color properties of the respective 3D subframes to detect a subframe of the data from 2D + D format depth as shown in Figure 5. The output of the correspondence calculation is converted into the CMP 607 scoring unit to a 610 format score for the SBS 3D format, for example, through normalization.
    Format scores for different 3D formats should be evaluated to determine the actual 3D format used, if any. The format scores can be compared to each other and / or to respective limits. A format score can express a confidence level indicative of the likelihood that the actual 3D subframes are present according to the respective 3D format. Format scores can be determined repeatedly, for example, every second, and the multiple measures and respective levels of confidence can be assessed in a weighted assessment process. The process can be terminated if a pre-established confidence level has been reached, and / or after a predetermined interval. The process may involve a majority of votes weighted with the confidence level, for example, a high MAD value with small differences between “subsequent assumptions for a specific 3D format has a low confidence for this format. If none of the 3D formats has a sufficient level of confidence, a 2D (monovideo) signal is assumed.
    In the practical realization shown in Figure 6, the MAD value must be low, and it is tested to be below a certain threshold THl in the CMP unit so that, if the score is reliable enough, make a decision that the first 3D format is present at the entrance , indicated by an H1l value of the format score for the first 3D format.
    Considering that the left and right images are taken from different points of view, it is best to eliminate this influence as much as possible, which can, for example, be done through the Disparity Estimate (DE) and applying the “motion / vision ”(MC) in the resulting R vision in E '. It is noticed that the motion estimation technology can be used here in subframes E and D, that is, to detect spatial differences instead of the temporal differences between the two frames.
    Motion estimation is well known and known algorithms can be similarly applied to estimate disparity.
    Other techniques for estimating disparity and vision compensation can also be used to determine E.
    The comparison section can therefore be provided with a 3D DE / MC 604 subframe processor to reduce differences between subframes based on the assumption that memory units E and D actually contain left and right subframes.
    For this purpose, the DE / MC processor applies an appropriate disparity estimate and / or motion compensation algorithm to the contents of D to generate a compensated frame E 'which corresponds to a compensated version of D indicated by E' = MC (D) . Subsequently, table E is compared to table E 'using the MAD correspondence calculation unit.
    The processor shown in Figure 6 has a second section in the parallel arrangement to simultaneously provide a second format score for a second 3D format.
    In the bottom section of the figure a video input signal 600 is coupled to a 3D subframe SPV 611 unit. The SPV unit has the function of vertically dividing the 2D frame in the input signal according to a top-to-bottom arrangement ( TB) of the 3D subframes.
    A T 612 memory unit stores pixels from the top section of the frame, for example, the left 3D subframe, and a D 613 memory unit stores pixels from the bottom section corresponding to the right 3D subframe.
    Subsequently the 3D subframes are coupled to a comparison section to calculate a correspondence between the 3D subframes.
    Another MAD 615 correspondence calculation unit is provided equal to the 605 unit described above.
    The corresponding calculation output is converted to another CMP 617 scoring unit, equal to the 607 unit described above, into a 620 format score for the 3D TB format.
    Optionally the score of the format can be directly compared to the TH2 limit in the CMP unit, if the score is reliable, it immediately makes a decision that the second 3D format is present at the entrance, charged with a Vl value of the format score for the second 3D format .
    The comparison section can be provided with a 3D DE / MC 614 subframe processor, equal to unit 604 described above, to reduce the differences between subframes, and / or a POL 616 polarity unit, equal to unit 606 described below, to determine a polarity score and generate a second left / right ED / DE polarity status signal.
    The function of the processor is to test the assumption that the input format received is SBS on the top branch of the diagram, or that the input format received is TB on the bottom branch of the diagram.
    If both assumptions are false (in the realization both H1 and Vl are false) then the input signal is apparently a regular 2D signal.
    In relation to the realization in Figure 6 having 2 parallel sections, it is noticed that other arrangements can be easily derived, such as other parallel sections to analyze other 3D formats, or a sequential arrangement where the same units are programmed according to different 3D formats sequentially to provide format scores for the respective 3D formats.
    In an accomplishment having sequential testing of multiple 3D formats, the determination of respective format scores for a number of 3D formats from the set of possible 3D formats is arranged in a predetermined order, and the 3D status signal is established when the evaluation of the scores format provides a predetermined level of confidence. Consequently, the 3D status signal is established when one of the 3D formats tested sequentially has obtained a reliable format score. The predetermined order can be used to increase the speed of detection, and can, for example, be based on a decrease in the probability of occurrence, a configuration by a user, and / or a configuration through a 3D content provider. For example, a content provider can establish the predetermined order in a set-top box based on the actual occurrence of 3D formats.
    In one embodiment the user can be offered with an option to correct the 3D status signal based on an order of the 3D format scores. The system first determines the most likely format, but if the result is not correct, the user can go to the next likely candidate based on the format scores with a button on the remote.
    In practical realizations the processor of the 3D subframe DE / MC 604 can be arranged for the pre-processing of the respective 3D subframes to increase the comparison as follows: - calculate an estimate of disparity between the 3D subframes and compensate for at least one of the 3D subframes with based on the disparity estimate before another comparison; and / or - calculate an auto correlation of the 2D frame to compare with a correlation of the 3D subframes.
    The correlation is determined by the MAD unit, and the output of the auto correlation can be used to establish the threshold TH1 as described above.
    It is noticed that in practice the subsampling can be applied first (that is, horizontal or vertical or hor / ver decimation) in the contents of E and D, which will reduce the computational complexity of the other units, such as the DE / MC and MAD units.
    In a realization of the specific type of the subsampling it can also be detected.
    For example, the SBS checkerboard format has a different subsampling than the normal SBS method.
    Distinguishing the SCB from the SBS can be based on the analysis of the spectrum where the SCB spectrum will be more shaped in diamond / intersection (symmetrical between vertical and horizontal), The vertical SBS in peak shape (the high horizontal frequencies more suppressed than the high frequencies vertical). In practice, a number of 3D formats can be detected in a different process as follows.
    An auto correlation for the total 2D frame is calculated (for example, based on MAD or another technique), and other correlations are subsequently calculated to determine format scores based on some well-chosen offsets such as: a.
    One and two pixels to the right (for CI format) b.
    One or two pixels down (for LI format) c.
    Pixel half frame to the right (for SBS format) d.
    Pixel one half frame down (for TB format) Second, format score values are compared to decide which format (2D, TB, LI, CI, CB) is most likely.
    Third, 2D + D is detected separately, for example, by determining a U / V constancy in the left / right half or in the top / bottom half of the image.
    It is noticed that 2D + D could be easily detected because all pixels in one half, the UV values of all pixels would be a fixed value, usually O (128). If this is the case for halves, it is obviously a black and white video source.
    In an alternative embodiment, to speed up processing, MAD or MSD for only the first column of pixels is calculated from the supposed 3D subframes, for example, using the left and right part for SBS.
    If they show a high correlation then the 3D format is likely to be correct.
    More columns can be added to improve reliability.
    An even faster approach is to compare only the average color of the first pixel column on the left and right of the image.
    If the SBS is not detected, the system continues by dividing the signal in the different parts, for TB the division is done in the horizontal direction, then again the same algorithm is used for LI the first pixels of the even and odd lines, for CL the columns etc.
    If none of these results in positive compatibility then the system reverts to 2D.
    It is noticed that several possible 3D formats have left and right 3D subframes [E and DJ] arranged in the 2D frame according to a left / right polarity.
    The left / right polarity can also be automatically detected for the respective 3D format based on the assumption of a predetermined depth distribution in the average video content, for example, using the disparity estimate in order to obtain a depth map.
    By analyzing this depth map, which is based on an assumption of polarity, it can be verified that the assumed polarity is correct.
    When the assumed polarity is correct, the depth at the bottom of the screen should indicate objects closer to the viewer and at the top should indicate objects farther away from the viewer.
    It is realized that polarity detection can also be applied independently of the automatic detection of the 3D format.
    For example, when 3D video content is received through a 3D distribution system and the 3D format has corresponding control signals, polarity detection can be applied to determine and verify polarity, for example, to ensure that no errors has been done in the storage, processing or transfer of 3D subframes.
    Figure 7A shows a depth map based on a correct left / right polarity.
    In the Figure a depth map is shown with dark pixels indicating great depth and light values indicating objects close to the viewer.
    The depth map can be generated by estimating the disparity and converting the disparity into depth values.
    In practice, to test the polarity, the base of the depth map can be generated in the heavily subsampled / decimation entry frames.
    Figure 7b shows a depth map based on the wrong left / right polarity.
    The disparity estimate can be applied to obtain the depth map.
    Through the analysis of the depth map it can be verified if the assumed polarity is correct.
    When the assumed polarity is correct, the depth values at the bottom of the screen should indicate objects close to the viewer and at the top they should indicate objects further away from the viewer (as is the case in Fig. 7a). When the assumed polarity is wrong, the depth values at the bottom of the screen should indicate objects further away from the viewer and the upper part should indicate objects closer to the viewer (as in Fig. 7b).
    In one embodiment, the processor section is provided with a POL 606 polarity unit to determine a polarity score based on a predetermined depth distribution occurring in the 3D frame. An ED / DE output left / right polarity status signal is generated to establish 3D status based on an evaluation of the polarity score, for example, based on a minimum difference between the average depth in the top half of the frame 3D and the average depth in the bottom half of the 3D frame. The depth values in the 3D frame can be directly available in a 2D + D format, or can be derived by the 3D subframe processor DE / MC 604 based on the disparity of 3D subframes E and D.
    In a practical embodiment, the determination of the polarity score is based on, for at least one of the possible polarity arrangements of the 3D subframes, determining whether the depth in the 3D frame increases with vertical height in the frame, or determining whether the depth at an edge vertical of a 3D subframe indicates a depth behind the screen, for example, objects or background. Similarly, determining the polarity score can be based on determining how the disparity values in the 3D frame change with the vertical height in the frame or at the edges.
    In practice, normal 3D video content without screen effects is relatively rare and concentrated in small parts of the image. Consequently, the overall average depth could be calculated as an indicator of polarity. It is noticed that the depth beyond the screen level implies disparity values in a certain horizontal direction, due to the displacement between the right and left 3D subframes. In a practical realization the disparity can be used instead of the actual depth. In addition, the disparity can be estimated similarly to motion, that is, to calculate the motion vectors between the left and right image using a well-known motion estimation algorithm. Due to the expected disparity / depth distribution, such "motion" vectors would have a preferred horizontal direction of motion. The polarity status signal is derived from said direction.
    In another embodiment, the polarity unit evaluates subframe E and D by applying a compression algorithm such as MPEG2 and determining which groups (blocks) of pixels can be encoded predicatively (P) or bidirectionally (B) (which corresponds to having vectors motion) or (1) encoded (which corresponds to having motion vectors). In fact, at certain edges of the 3D subframes the number of encoded pixels I can deviate from the mean, whose deviation can be used to indicate polarity. Normally more pixels I should be on the left side of frame E (the part that is cut in frame D) and the right side of frame D. Consequently the number of pixels encoded I at the edges of the 3D subframes is used to decide the left polarity / right. It is noticed that the 3D format can also be detected based on the II pixels. When pixels I tend to appear on a vertical axis in the center of the 2D frame, this is a strong indication of a 3D SBS signal. When pixels I tend to appear on a horizontal axis in the center of the 2D frame, this is a strong indication of a 3D TB signal.
    Figure 8 shows an analysis of depth in the horizontal fields.
    The Figure shows a depth map in an assumed left / right polarity, which must be tested for correction.
    The depth map is subdivided into a number of horizontal fields 81, 82, 89, also called horizontal boxes.
    By dividing the depth map in the horizontal boxes, the average depth value in each box can be calculated.
    Regression analysis is applied to the average values of the boxes to determine if the bottom is darker then the top of the reverse and to determine the polarity.
    Figure 9 shows an analysis of depth in the vertical fields.
    The Figure shows a depth map based on an assumed left / right polarity, which is to be tested for correction.
    The depth map is subdivided into a number of vertical fields 91, 92, also called vertical boxes.
    By dividing the depth map into vertical boxes, each regression analysis of the box can be applied and for each box to determine the lower part is darker then the upper part in the opposite direction.
    If most boxes correspond to the correct polarity assumption, it can be safely assumed that the polarity is correct, otherwise the polarity is reversed.
    Whether they are almost equal to the result of the ED analysis is uncertain, and another video input must be analyzed.
    Other alternatives for determining the polarity that does not require a depth map are trapezoid detection or edge detection.
    Trapezoid detection involves the following steps - Assume that the 1st frame is E, and the 2nd frame is D.
    —- Estimate the disparity or depth of the E + D frames - If the disparity / depth increases as you move from the bottom the assumption is correct, otherwise rotate E, D - In a variant, restrict the search area to the top of frames E and D (tentative), and check that the disparity / depth is positive The detection of the edge involves the following steps - Assume that the 1st frame is E, the 2nd frame is D - Try to match the next region to the right edge from the screen in frame E to the right edge of frame D (determination of the vector P) - If there is no good match, the assumption is correct, otherwise rotate E and D - This procedure can be repeated on the left edge of the frames with E and D reversed.
    The idea is that the content of the edges is behind the screen (or at least there is a float window next to it), so that the part of the D-eye of an object near the right edge of the screen will be obscured, thus its part E eye cannot be matched.
    Similarly, the E-eye part of an object near the left edge of the screen.
    In one embodiment, to improve the reliability of polarity and / or shape detection, scene cut detection is applied.
    Thus, processor 52 is arranged to detect scene changes.
    The 3D format detection is performed multiple times for multiple scenes, and the status signal is finally established based on the format detection in at least two different scenes.
    Consequently, determining the respective format score involves detecting a scene change in the video content, and calculating format scores for at least two scenes.
    In practice
    3 scenes can be used, for example, when calculating 3 subsequent scene decisions for 3 consecutive parts of video marked by cut scenes, at least two format decisions must be consistent and at most one must be ensured.
    A 2D expansion device can be defined as follows. The video device for processing a video signal, the device comprising receiving means for receiving the video signal comprising video data representing either the three-dimensional [3D] video content formatted according to a 3D format or two-dimensional [2D] video content ] formatted according to a 2D format, the 3D format having at least two 3D subframes to constitute a single 3D frame, the 2D format having a 2D frame, and the device comprising a processor to detect the presence of said 3D content, and to convert the 3D content to a 2D video output signal by retrieving at least one 3D subframe and expanding the subframe to the 2D frame according to the 2D format. The 2D expansion device detects 3D content at the input, and generates either a 2D or 3D video signal based on the required output mode. Advantageously a single 3D broadcast signal can be used, to also provide a 2D version of the video content for 2D viewers. In addition, 3D content can be transferred in a 2D signal format as explained above to be compatible with existing distribution systems. Due to the 2D expansion function, the 2D viewer can still be provided with a 2D video signal.
    In one embodiment, the 2D expansion device is a video device as shown in Figure 1, having a 2D expansion function arranged as follows. Processor 52 is arranged to detect the presence of 3D video content according to a 3D format in the video input signal on input unit 51. In addition, the device is arranged to provide a 2D output signal by converting the content from 3D video at the input for the 2D output signal. Thus, a part of the 3D input signal, for example, a left frame is expanded to the 2D frame in the 2D output signal, while at the same time removing the original 3D content. 3D control data (if any) can also be removed from the signal and replaced with 2D control data.
    The 2D output signal can be coupled to a 2D video device to be connected instead of the 3D video device 60, or can be selected by the user to avoid intentionally displaying the 3D video. The 2D expansion device may allow the user to select a 2D output mode or a 3D output mode to match the output signal to the user's requirements, for example, to match the equipment the user intends to connect or has connected. Alternatively, or in addition, the 2D expansion device may be arranged to exchange control data with a video device coupled to output unit 55, for example, according to HDMI as described above. The control data can indicate the 3D capability of a connected display device, and the video device can automatically select the 2D or 3D output mode according to said display capability.
    Processor 52 is arranged to convert the 3D video content at the input to the 2D output signal, if necessary. The conversion is triggered based on the presence of the 3D video content in the input mode and in the 2D output mode having been established. The processor first determines the 3D format of the input signal. It should be noted that the 3D format can be automatically detected from the video data as described above, or it can be derived from a control signal provided with the input signal.
    In a realization of the 2D expansion device, a content provider may include a dedicated control signal in the 3D video signal for the 2D expansion device to indicate the presence of the 3D content that must be converted and / or the specific 3D format of the video signal, for example, side-by-side 3D format as shown in Figure 3. Consequently, SBS or TB signaling can be included in the video stream.
    Subsequently, the processor retrieves a 3D subframe, for example, a left 3D subframe from the left part of the frame in the input video signal according to SBS.
    The 3D subframe can be reduced in size when compared to the 2D output frame, and therefore the video data from the 3D subframe must be expanded to the size of the 2D frame and inserted into the output signal.
    For SBS the horizontal size must be expanded, while the vertical size (number of lines) can remain the same.
    Consequently, the conversion involves generating the new 2D output frame having the necessary resolution, for example, through the interpolation of the missing pixels, or any suitable way to increase the sampling.
    In a realization of the 2D expansion device, no 3D output mode is provided, and the conversion is applied to any 3D video content detected at the input.
    In practice, such a device would be very suitable for legacy users of 2D video equipment, such as ordinary 2D TV sets coupled to a cable set-top box or satellite signals.
    Such legacy set-top box can be modified in the 2D expansion box through a software update, which may be possible under the control of the content provider, or through an update process initiated by the user. Advantageously, the content provider does not have to transmit the same content twice, that is, once in 3D for new users equipped with a new set box and a 3D display, and separately, in an additional channel, also in 2D. Only a single transmission of the new 3D format signal would be sufficient, because legacy 2D display devices would automatically receive the expanded version of the 2D expansion device, that is, the modified set-top box. It should be noted that the 2D expansion device can also contain any of the units and / or functions as described above for automatic detection of the 3D format in a 2D format signal. The 3D status signal provided through automatic detection now controls the function of 2D expansion.
    One embodiment is a video device for processing a video signal, the device comprising receiving means for receiving the video signal comprising video data representing either the three-dimensional [3D] video content formatted according to a 3D format or video content two-dimensional [2D] formatted according to a 2D format, the video signal having a 2D frame and a control structure of a 2D format, the 3D format having at least two 3D subframes to constitute a single 3D frame and one of a set of possible 3D formats, a processor to provide a 3D status signal indicative of the video signal format, the processor being arranged to determine a respective format score for at least one of the possible possible 3D formats by processing the video data accordingly with the respective properties of the predetermined format to derive and compare the respective 3D subframes, and establish the 3D status signal based on a evaluation of the format score to indicate the format of the video signal.
    Optionally, determining the respective format score can comprise calculating a correspondence between the 3D subframes using at least one of - calculating an average of absolute differences between the 3D subframes; - calculate color properties of the respective 3D subframes to detect a depth data frame.
    Optionally, determining the respective score of the format can comprise detecting a scene change in the video content, and calculating a match for at least two scenes.
    Optionally, comparing the respective 3D subframes can comprise at least one of - calculate an estimate of disparity between the 3D subframes and compensate for at least one of the 3D subframes based on the disparity estimate before another comparison; - calculate an auto correlation of the 2D frame to compare with a correlation of the 3D subframes.
    Optionally, where at least one of the possible 3D formats comprises left and right subframes [E and DJ] arranged in the 2D frame according to a left / right polarity, the processor can be arranged to, when determining the format score, determine a score polarity based on a predetermined depth distribution occurring in the 3D frame, and establishing 3D status includes establishing a left / right polarity status signal based on an evaluation of the polarity score.
    Optionally, the processor may have detector means to compare the respective 3D subframes by detecting at least one of - a vertical black mask on a vertical border of the 3D subframes; - a horizontal black mask on a horizontal border of the 3D subframes.
    It will be appreciated that in the description above, for clarity, realizations of the invention have been described with reference to the different functional units and processors.
    However, it will be apparent that any suitable distribution of functionality between different functional units or processors can be used without diminishing the invention.
    For example, the illustrated functionality to be performed by separate units, processors or controllers can be performed by the same processor or controllers.
    Consequently, references to specific functional units are only to be seen as references to appropriate means to provide the described functionality rather than indicative of a strictly logical or physical organization or structure.
    The invention can be implemented in any suitable form including hardware, software, firmware or any combination thereof.
    The invention can optionally be implemented at least partially as computer software running on one or more data processors and / or digital signal processors.
    The elements and components of an embodiment of the invention can be physically, functionally and logically implemented in any suitable form.
    In fact, the functionality can be implemented in a single unit, in a plurality of units or as part of other functional units.
    In this way, the invention can be implemented in a single unit or it can be physically and functionally distributed among the different units and processors.
    Although the present invention has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein.
    Instead, the scope of the present invention is limited only by the accompanying claims.
    In addition, although an aspect may appear to be described in connection with particular embodiments, a person skilled in the art would recognize that various aspects of the embodiments can be combined according to the invention.
    In the claims, the term comprising does not exclude the presence of other elements or stages.
    In addition, although listed individually, a plurality of means, elements or steps of the method can be implemented through, for example, a single unit or processor.
    In addition, although the individual aspects may be included in different claims, they can possibly be advantageously combined, and the inclusion in the different claims does not imply that a combination of aspects is not possible and / or advantageous.
    Also, the inclusion of an aspect in a category of claims does not imply a limitation for this category, but instead indicates that the aspect is equally applied to other categories of claim as appropriate.
    Furthermore, the order of the aspects in the claims does not imply any specific order in which the aspects are to be worked on and in particular the order of the individual steps in a method claim does not imply that the steps must be performed in that order.
    Instead, the steps can be performed in any suitable order.
    Additionally, singular references do not exclude a plurality.
    Thus, the references “one”, “one”, “first (a)”, “second (a)” etc. it does not exclude a plurality.
    Reference signs in the claims are provided merely as an example of clarification should not be construed as limiting the scope of the claims in any way.
    权利要求:
    Claims (13)
    [1]
    1. VIDEO DEVICE (50) FOR PROCESSING A VIDEO SIGNAL, the device comprising receiving means (51, 58, 59) for receiving the video signal comprising video data representing three-dimensional [3D] formatted video content of according to a 3D format or two-dimensional [2D] video content formatted according to a 2D format, the 3D format having at least 3D subframes to constitute a single 3D format and one of a set of possible 3D formats, a processor (52 ) to provide a 3D status signal indicative of the video signal format, the processor being arranged to determine respective format scores for a number of possible 3D formats by processing the video data according to the respective predetermined format properties for derive and compare the respective 3D subframes that determine the respective format scores for said number of possible 3D formats is arranged in a predetermined order finished, and establish the 3D status signal based on an evaluation of the respective format scores to indicate the format of the video signal, when the evaluation of the format scores provides a predetermined level of confidence, characterized in that the video signal has a 2D frame and a control structure of a 2D format, the 3D subframes being generated by a spatial subsampling format and the photo elements subsampled from the 3D subframes being arranged in the 2D frame of the video signal.
    [2]
    2. VIDEO DEVICE, according to claim 1, characterized in that the set of possible 3D formats comprises at least one of - a side-by-side format [SBS] having the subframes
    3D arranged side by side in a 2D frame; - a top-to-bottom [TB] format with the 3D subframes arranged at the top and bottom of a 2D frame; - an interleaved line format [L1] with the 3D subframes arranged interleaving the lines of the 3D subframes in a 2D frame; - an interleaved column format [Cl] with the 3D subframes arranged through the interspersed columns of the 3D subframes in a 2D frame; - an interlaced checkerboard format [CB] with the 3D subframes arranged through subsampling pixels of the 3D subframes in a checkerboard pattern and interleaving the subsampled pixels in a checkerboard pattern in a 2D frame; - a side-by-side data board format [CBS] with the 3D subframes arranged through subsampling pixels of the 3D subframes in a checkerboard pattern and the subsampled pixels arranged in the 3D subframes side by side in a 2D frame; - a 2D depth format [2D + D] having a 2D subframe and a depth data subframe like the 3D subframes arranged in a 2D frame; and the processor is arranged to derive the 3D subframes of the video signal for the respective 3D format.
    [3]
    3. - VIDEO DEVICE, according to claim 1, characterized in that the predetermined order is based on at least one of: - a decrease in the probability of occurrence; - a configuration through a user; - a configuration through a 3D content provider.
    [4]
    4, VIDEO DEVICE, according to claim 1, characterized in that the determination of the respective format score comprises calculating a correspondence between the 3D subframes through at least one of: - calculating a correlation between the 3D subframes; - calculation of an average of absolute differences between 3D subframes; - calculation of the color properties of the respective 3D subframes to detect a depth data subframe.
    [5]
    5. VIDEO DEVICE, according to claim 1, characterized in that the determination of the respective format score comprises the detection of a scene change in the video content, and calculating a correspondence for at least two scenes.
    [6]
    6. VIDEO DEVICE, according to claim 1, characterized in that the comparison of the 3D subframes comprises at least one of: - calculation of a disparity estimate between the 3D subframes and compensating for at least one of the 3D subframes based on the estimate disparity before another comparison; - calculation of a 2D table auto-correlation to compare with a correlation of the 3D subframes.
    [7]
    7. VIDEO DEVICE, according to claim 1, characterized in that at least one of the possible 3D formats comprises left and right 3D subframes [E and D] arranged in the 2D frame according to a left / right polarity, and the processor (52) is willing to, when determining the format score, determine a polarity score based on a predetermined depth distribution occurring in the 3D frame, and establishing 3D status includes establishing a left / right polarity status sign based on in an evaluation of the polarity score.
    [8]
    8. VIDEO DEVICE, according to claim 7, characterized in that the determination of the polarity score comprises, at least one of the possible polarity arrangements of the 3D subframes, - determining whether the depth in the 3D frame increases with the vertical height in the painting; - determine whether the depth at a vertical edge of a 3D subframe indicates a depth behind the screen.
    [9]
    9. VIDEO DEVICE, according to claim 1, characterized in that the device comprises at least one of the transmission means (55) for transmitting a 3D display signal (56), the 3D display signal comprising the content of 3D video and control data indicative of the 3D status signal; - in the receiving means, read the means (58) to read a recording carrier for receiving the video signal; - a 3D display (63) for transforming a 3D signal based on the 3D status signal.
    [10]
    10. VIDEO DEVICE, according to claim 1, characterized in that the processor has detector means (53) for comparing the respective 3D subframes by detecting at least one of - a vertical black mask on a vertical edge of the 3D subframes; - a horizontal black mask on a horizontal border of the 3D subframes.
    [11]
    11. VIDEO SIGNAL PROCESSING SYSTEM, characterized in that it comprises the video device as claimed in claim 10, and a video signal comprising video data or representing three-dimensional [3D] video content formatted according to a 3D format or content two-dimensional [2D] video formatted according to a 2D format, the video signal having a 2D frame and a control structure of a 2D format, the 3D format having at least two 3D subframes to constitute a single 3D frame and being one of a set of possible 3D formats, video data having at least one of - a vertical black mask on the vertical edges of the 3D subframes, while the aspect ratio of video content does not require vertical black bars; - a horizontal black mask on the horizontal edges of the 3D subframes, while the aspect ratio of video content does not require horizontal black bars; to allow detection of the black mask to determine the 3D format.
    [12]
    12. METHOD OF PROCESSING A VIDEO SIGNAL, the method comprising - receiving the video signal comprising video data representing either the [3D] three-dimensional video content formatted according to a 3D format or two-dimensional [2D] formatted video content according to a 2D format, the 3D format having at least two 3D subframes to constitute a single 3D frame and one of a set of possible 3D formats, —- provide a 3D status indicative of the video signal format based on - - determination of the respective “format scores for a number of possible 3D formats by processing the video data according to the respective predetermined format properties to derive and compare the respective 3D subframes, which determine the respective format scores for the said number of possible 3D formats is arranged in a predetermined order, and —- establishment of 3D status based on an assessment of the respective points Use of format to indicate the format of the video signal, when the evaluation of the format scores provide a predetermined level of confidence, characterized in that the video signal has a 2D frame and a control structure of a 2D format, the 3D subframes being generated through a spatial subsampling format and photography elements subsampled from the 3D subframes being arranged in the 2D frame of the video signal.
    [13]
    13. COMPUTER PROGRAM PRODUCT FOR PROCESSING A VIDEO SIGNAL, characterized in that the program is operative to make a processor perform the respective steps of the method as claimed in claim 12.
    类似技术:
    公开号 | 公开日 | 专利标题
    BR112012019612A2|2020-07-14|video device for processing a video signal, video signal processing system, method of processing a video signal and computer program product for processing a video signal
    US8488869B2|2013-07-16|Image processing method and apparatus
    RU2554465C2|2015-06-27|Combination of 3d video and auxiliary data
    US8730307B2|2014-05-20|Local multi-view image display apparatus and method
    US8798160B2|2014-08-05|Method and apparatus for adjusting parallax in three-dimensional video
    BRPI0911016B1|2021-01-05|three-dimensional image signal provision method, three-dimensional image signal provision system, signal containing a three-dimensional image, storage media, three-dimensional image rendering method, three-dimensional image rendering system to render a three-dimensional image
    US20100220920A1|2010-09-02|Method, apparatus and system for processing depth-related information
    US20110298898A1|2011-12-08|Three dimensional image generating system and method accomodating multi-view imaging
    Winkler et al.2013|Stereo/multiview picture quality: Overview and recent advances
    EP2553932B1|2018-08-01|Disparity value indications
    WO2010064118A1|2010-06-10|Methods and systems for presenting three-dimensional motion pictures with content adaptive information
    KR101066550B1|2011-09-21|Method for generating vitual view image and apparatus thereof
    TWI651960B|2019-02-21|Method and encoder/decoder of encoding/decoding a video data signal and related video data signal, video data carrier and computer program product
    CN104144334A|2014-11-12|Subtitle detection for stereoscopic video contents
    US9639944B2|2017-05-02|Method and apparatus for determining a depth of a target object
    US20130093843A1|2013-04-18|Method for configuring stereoscopic moving picture file
    US8908010B2|2014-12-09|Method for processing video input by detecting if picture of one view is correctly paired with another picture of another view for specific presentation time and related processing apparatus thereof
    Ramachandran et al.2012|Multiview synthesis from stereo views
    KR20110025020A|2011-03-09|Apparatus and method for displaying 3d image in 3d image system
    Knee2012|Getting machines to watch 3d for you
    EP2587812A2|2013-05-01|Method for configuring stereoscopic moving picture file
    Lü et al.2010|Virtual view synthesis for multi-view 3D display
    Zhang et al.2011|Automatic 3D video format detection
    Theodoridis et al.2015|Object motion analysis description in stereo video content
    US20180124373A1|2018-05-03|Method for providing continuous motion parallax effect using an auto-stereoscopic display, corresponding device, computer program product and computer-readable carrier medium
    同族专利:
    公开号 | 公开日
    TW201143368A|2011-12-01|
    CN102742283B|2016-04-27|
    US20120314028A1|2012-12-13|
    RU2012138335A|2014-03-20|
    WO2011098936A3|2011-12-01|
    RU2568309C2|2015-11-20|
    US9325964B2|2016-04-26|
    US9729852B2|2017-08-08|
    EP2534844A2|2012-12-19|
    KR20120125345A|2012-11-14|
    TWI536801B|2016-06-01|
    JP5894537B2|2016-03-30|
    CN102742283A|2012-10-17|
    EP3258688A2|2017-12-20|
    US20170085859A1|2017-03-23|
    KR101801017B1|2017-11-24|
    JP2013519286A|2013-05-23|
    WO2011098936A2|2011-08-18|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    JPH0471042A|1990-07-12|1992-03-05|Toshiba Corp|Test supporting system based upon execution trace|
    TW344931B|1994-12-08|1998-11-11|Matsushita Electric Ind Co Ltd|Average luminance level detection apparatus and aspect ratio auto-discrimination apparatus for a television signal using the same|
    JPH09271042A|1996-03-31|1997-10-14|Sony Corp|Stereoscopic vision method and device|
    JPH10191394A|1996-12-24|1998-07-21|Sharp Corp|Multi-view-point image coder|
    EP1024672A1|1997-03-07|2000-08-02|Sanyo Electric Co., Ltd.|Digital broadcast receiver and display|
    JPH10257525A|1997-03-07|1998-09-25|Sanyo Electric Co Ltd|Digital broadcast receiver|
    US7176960B1|1999-09-20|2007-02-13|The Trustees Of Columbia University In The City Of New York|System and methods for generating spherical mosaic images|
    KR101016973B1|2001-10-02|2011-02-25|코닌클리케 필립스 일렉트로닉스 엔.브이.|Method and apparatus for recording a digital information signal|
    CA2380105A1|2002-04-09|2003-10-09|Nicholas Routhier|Process and system for encoding and playback of stereoscopic video sequences|
    EP1431919B1|2002-12-05|2010-03-03|Samsung Electronics Co., Ltd.|Method and apparatus for encoding and decoding three-dimensional object data by using octrees|
    KR100556848B1|2003-05-13|2006-03-10|엘지전자 주식회사|Apparatus and method for encoding and decoding moving picture using digital water marking|
    JP4190357B2|2003-06-12|2008-12-03|シャープ株式会社|Broadcast data transmitting apparatus, broadcast data transmitting method, and broadcast data receiving apparatus|
    JP2005311983A|2004-04-26|2005-11-04|Olympus Corp|Image processing apparatus|
    CN101006732B|2004-08-17|2010-12-29|皇家飞利浦电子股份有限公司|Detection of view mode|
    JP2006195018A|2005-01-12|2006-07-27|Nippon Telegr & Teleph Corp <Ntt>|Three dimensional display method, image generation side apparatus and image display side apparatus|
    JP2006332985A|2005-05-25|2006-12-07|Sharp Corp|Stereoscopic image format decision device|
    TWI323610B|2006-09-14|2010-04-11|Novatek Microelectronics Corp|Apparatus and method for video de-interlace|
    JP4662169B2|2006-09-27|2011-03-30|ソニー株式会社|Program, detection method, and detection apparatus|
    US8749615B2|2007-06-07|2014-06-10|Reald Inc.|Demultiplexing for stereoplexed film and video applications|
    KR101450145B1|2007-06-11|2014-10-13|삼성전자 주식회사|Method and apparatus for generating header information of stereoscopic image data|
    EP2501137A3|2007-06-11|2012-12-12|Samsung Electronics Co., Ltd.|Method and apparatus for generating header information of stereoscopic image|
    KR101386810B1|2007-09-07|2014-04-17|삼성전자주식회사|Apparatus and method for identifying three-dimensional images|
    WO2009077929A1|2007-12-14|2009-06-25|Koninklijke Philips Electronics N.V.|3d mode selection mechanism for video playback|
    KR101539935B1|2008-06-24|2015-07-28|삼성전자주식회사|Method and apparatus for processing 3D video image|
    KR20100138806A|2009-06-23|2010-12-31|삼성전자주식회사|Method and apparatus for automatic transformation of three-dimensional video|
    EP3258688A2|2010-02-09|2017-12-20|Koninklijke Philips N.V.|3d video format detection|JP2008106185A|2006-10-27|2008-05-08|Shin Etsu Chem Co Ltd|Method for adhering thermally conductive silicone composition, primer for adhesion of thermally conductive silicone composition and method for production of adhesion composite of thermally conductive silicone composition|
    EP3258688A2|2010-02-09|2017-12-20|Koninklijke Philips N.V.|3d video format detection|
    GB2483637A|2010-09-10|2012-03-21|Snell Ltd|Detecting stereoscopic images|
    US9584798B2|2010-12-09|2017-02-28|Google Technology Holdings LLC|Method and apparatus for managing 3D video content|
    WO2012143754A1|2011-04-19|2012-10-26|Institut für Rundfunktechnik GmbH|Method and system for decoding a stereoscopic video signal|
    GB2491102B|2011-05-17|2017-08-23|Snell Advanced Media Ltd|Detecting stereoscopic images|
    US8913104B2|2011-05-24|2014-12-16|Bose Corporation|Audio synchronization for two dimensional and three dimensional video signals|
    WO2012170984A1|2011-06-09|2012-12-13|Cstereo3D, Llc|Device and method for rendering and delivering 3-d content|
    FR2973980A1|2011-08-25|2012-10-12|Sagemcom Broadband Sas|Method for detection of composite format utilized to format sequence of three-dimensional video, received by decoder, involves utilizing format to format sequence of images if comparison between threshold and correlation satisfies condition|
    US9185398B2|2011-09-22|2015-11-10|Google Technology Holdings LLC|Method and apparatus for providing three-dimensional content|
    KR101328846B1|2011-12-06|2013-11-13|엘지디스플레이 주식회사|Stereoscopic image display device and driving method thereof|
    CN103179426A|2011-12-21|2013-06-26|联咏科技股份有限公司|Method for detecting image formats automatically and playing method by using same|
    WO2013100350A1|2011-12-28|2013-07-04|Samsung Electronics Co., Ltd.|Image processing apparatus, upgrade apparatus, display system including the same, and control method thereof|
    JP5817639B2|2012-05-15|2015-11-18|ソニー株式会社|Video format discrimination device, video format discrimination method, and video display device|
    TWI630815B|2012-06-14|2018-07-21|杜比實驗室特許公司|Depth map delivery formats for stereoscopic and auto-stereoscopic displays|
    CN102821298B|2012-08-27|2015-06-17|深圳市维尚视界立体显示技术有限公司|Method, device and equipment for 3Dplaying adjustment and self adaptation|
    CN102857779A|2012-09-12|2013-01-02|康佳集团股份有限公司|Method and system for automatically identifying 3Dvideo signals by cloud server|
    CN102905157B|2012-09-29|2015-02-18|四川长虹电器股份有限公司|Television three-dimensionalsignal format automatically identifying method|
    EP2936808A1|2012-12-19|2015-10-28|Thomson Licensing|Methods and apparatus for automatically detecting image/video resolution and its color subsampling|
    CN103888839B|2012-12-20|2017-03-29|深圳市快播科技有限公司|A kind of video broadcasting method and equipment|
    TW201428675A|2013-01-08|2014-07-16|Pixart Imaging Inc|Video generating system and related method thereof|
    TWI498854B|2013-01-18|2015-09-01|Chunghwa Picture Tubes Ltd|Method of auto-determination a three-dimensional image format|
    CN103152535B|2013-02-05|2016-02-03|华映视讯（吴江）有限公司|The method of automatic decision 3D image format|
    KR101802582B1|2014-11-24|2017-11-28|타이완 세미콘덕터 매뉴팩쳐링 컴퍼니 리미티드|Layout optimization for integrated circuit design|
    CN104113745A|2013-04-17|2014-10-22|咏传电子科技（上海）有限公司|Display device and image display method thereof|
    US9092658B2|2013-04-25|2015-07-28|Nvidia Corporation|Automatic detection of stereoscopic content in video/image data|
    TWI502545B|2013-06-25|2015-10-01|Method of storing a content of a three-dimensional image|
    TWI602145B|2013-10-02|2017-10-11|國立成功大學|Unpacking method, device and system of packed frame|
    TWI602144B|2013-10-02|2017-10-11|國立成功大學|Method, device and system for packing color frame and original depth frame|
    KR101679122B1|2013-10-02|2016-11-23|내셔날 쳉쿵 유니버시티|Method, device and system for packing and unpacking color frame and original depth frame|
    CN104657966A|2013-11-19|2015-05-27|江苏宜清光电科技有限公司|3D format analysis method|
    US20150215530A1|2014-01-27|2015-07-30|Microsoft Corporation|Universal capture|
    TWI579594B|2014-12-29|2017-04-21|詠巨科技有限公司|3d display layer, 3d display structure and manufacturing method thereof|
    CN104994374B|2015-07-03|2017-01-18|宁波易维视显示技术有限公司|Method for automatically detecting three-dimensional format of video|
    US9767363B2|2015-10-30|2017-09-19|Google Inc.|System and method for automatic detection of spherical video content|
    WO2018049631A1|2016-09-14|2018-03-22|深圳市柔宇科技有限公司|Playback device and playback method thereof|
    CN108133210A|2017-12-12|2018-06-08|上海玮舟微电子科技有限公司|A kind of picture format recognition methods and device|
    EP3734970A4|2019-01-08|2020-11-11|Samsung Electronics Co., Ltd.|Method for processing and transmitting three-dimensional content|
    法律状态:
    2020-07-21| B15K| Others concerning applications: alteration of classification|Free format text: A CLASSIFICACAO ANTERIOR ERA: H04N 13/00 Ipc: H04N 13/156 (2018.01), H04N 13/183 (2018.01), H04N |
    2020-07-28| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|
    2020-09-01| B25D| Requested change of name of applicant approved|Owner name: KONINKLIJKE PHILIPS N.V. (NL) |
    2020-09-24| B25G| Requested change of headquarter approved|Owner name: KONINKLIJKE PHILIPS N.V. (NL) |
    2020-11-24| B11B| Dismissal acc. art. 36, par 1 of ipl - no reply within 90 days to fullfil the necessary requirements|
    2021-11-03| B350| Update of information on the portal [chapter 15.35 patent gazette]|
    优先权:
    申请号 | 申请日 | 专利标题
    EP10152997|2010-02-09|
    EP10152997.2|2010-02-09|
    PCT/IB2011/050455|WO2011098936A2|2010-02-09|2011-02-02|3d video format detection|
    [返回顶部]